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h Introduction 

Over tlie past decade, rese^l^rs have been concerned with the probJem of designing 
suitable Interfaces for the compul^ user (Bennett, 1972; Nickerson, Elkind & Carbonell, 196 ^; 
Rouse, 1975). On« major hurdle facing the design of good user-oriented systems has been 
the lack of a suitable and general representation of the user interface^! such a 
representation would describe a variety of user environments and serve as a communications 
toot between the human factors psychologist and the syetems designer. In this paper, 1 
present a methodology for representing person-computer Interfaces and for analyzing 
interface difficulties from this represent aion. To build my representation, I perform a tasK 
analysis and express the user's interaction with the computer system as an augmented 
transition nelworls CATN) grammar {Woods, 1970). 

I perform this task analysis to build a description of the possible streams of behavior that a 
user might exhibit in a given task environment when performing a specific task. 1 build this 
description by performing grammar induction on experimental streams of behavior obtained 
from a task which represents the task' environment under consideration. The approach Is 
almllar to Anderson's (1977) work with US except that I do not conceive of the resulting 
representation as a meaningful grammar. An iterative technique of prediction and 
experimentation is used to tune the representation! however, even with a first order model, 
the effects of the task environment on the user can be hypothesized from this structured 
view. The effect-* can be listed as desirable oi undesirable and the task environment 
modified to optimize the desirable interface effects. This representation of a user's behavior 
can also indicate what modifications to the task environment will have a negligible effect, 
r" 

In the second section of this paper, 1 present a simple user task and its accompanying 
. representation. Following this overview, I give a detailed set of steps 'describing how the 
task analysis Is performed. I then describe the computer system, ZOG,^ on which I test my 
mathodology. In the fifth section, I apply the task analysis techniques to a computer-aided 
instruction task, built In ZOa Following the application of the task analysis on a test- case, 1 
discuss the method's generality and stiortcomings. I a«so compare it briefly to the standard 
regression methods being used to study the user interface. 
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SU An Exampta: k letter Creallon Task 

In this section ! give »n example of what the finished product, the tasK analysis might look 
like. 4 heve eppUed my technique to the task of typing a letter. The operations required to 
create the final finished version of the letter can be either motor movements of the typ'st, 
cognitive decisions on the part of the typist, or operations executed by the typewriter. The 
motor movements of the typist are such operations as the striking of a key or the space bar. 
The cognitive operations are the choices made to lay out the letter on the page according to 
some desired format. Typewriter operations constitute such Items as bell ringing at the end 
of a long line or locking the keyboard. A sequence of these operations In the correct order 
generates the desired tetter. 

The task analysis for the typing ta^k consists of breaking up the task Into the recurring 
pperatlons that form the task. These subunits are catled task eUmmnts or simply, e/em«nt#. 
Tf»y will fall Into four categories! goal, cognUkttf motor, or inttrfac9 elements. 

Goal elements are those dictated by the nature of the task, Itself. In the letter typing task, 
goat elements are the requirements that the letter have a date, address, salutation, body and 
closing. The goal elements for the letter typing task are: 

<g«nerate date> 
<generate addr«ss> 
<f enerate sa{utation> 
<eenerafe body of latt«r> 
<generate closinc> 
<Senerate ?wtations> 

Goal elements govern the entire task. Motor and cognitive elements describe how the goal 
•laments are executed. A combination of motor, cognitive and Interface elements In the 
correct order form the goal elements. Examples of motor elijments are: 

<put paper In carriage) 
<hH carriage return) 
<s«t spacing to I > 
<typ« character) 
<af rike space bar) 

Cognitive elements are those parts of the task which reouire problem solving behavior on the 
part of the Individual executing the task. They are reflected in the verbal or motor behavior 
that results from these Internal states. Examples of cognitive elements In the typing task are 
such Items as: 
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<scift K«m in •ddrcsf tfst> 
<eh«ck If lt«m it dasircd ntmt> 
<eh«ck if ifMcing " 10 limO 
<r«tri»v« <odiy'B diU from m«mory> 

<ditcGv*r iirror> 

the cognitive elements govern the execution of the motor elements, e.g., spacing Is 
discontinued by a cognitive check on the number of spaces already generated. 

Tfie Interactive elements are those parts of tfte task execution which come from the 
environment but control what cognitive elements might follow. In the case of the typing task, 
thoy can be such simple indicators to the typist as: 

<betl ringing warning margln*s end) 
<fceyl>oard tocfc> 

<!ight print Indleating ribl>on ehanea> 

In the typewriter task, these form a negligible part of the task analysis. With a more mutable 
environment such as a computer interface, the interactive elements form a major part of the 
task. They prompt what cognitive elements will be executed In reaction to the Information 
they provide. 



Insert Table I about here 

Table 1 summarizes the types of task elements and thei. ipresentatlons In the typing task. 
Their ordering is non-deterministic; however, several general heuristics can be applied to 
their sequencing. 

X. a <g0f • alemant> subsumes Cfntaractive tlemants>, <cognltfva elamants), and <motor 
etemants). 

2. a <cognltlve elem«nt> follows a <eoBnltlva fjlamtnt> or an <lnteractiva elamant^ 

3. a <motor element) follows a <iytQtor element) or a <eognltiva element). 

4. an <lnteraclive element) follows a <motor element) or a <cogniiWe element). 

The te«K of typing a letter can now be broken into its task elements using the element types 
Meted In Table I. This task representation is presented as an ATM grammar describing ihm 
sequences of elements that can occur within a given context. Figure 1 Is an Illustration of 
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Goal Elementa 



<lnf art letter> 
<genar»te tatief> 
<ger^r«te data> 



^•nsrate address> 
<3ener«te $aiutatton> 
generate body> 
generate ciosing> 
<^«nerate rnstations^ 
<retnove letter> 



<type character> 
<hlt carrbge reiurn> 
<striKa space bar> 
<roll carrrage> 
<set spacmg> 
<typ« tab> N > 
<type backspace> 



<retriove today's date from memory> 
<dec!de to begin new Une> 
<recognize that character is mlstyped> 
<decide to stop spacing> 
<decide to begin new page> 
<tcan item in address list> 
<cHecK if item is desirecl name> 



<lock keyboard> 
<ring bell> 
<raturn carrlage> 



Table 1. Elements which form the components of a letter typing task- 



Moto*' Elemenis 



Cognlilve Elements 



Interface Elementa 
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iiow th« ATN grammar describes the typing task. Aw ATN grammar is used because It eltows 
on« to incorporate semantics in the behavior description. These semantics are instrumental In 
understanding those areas of the representation which are potentiai troubie spots. 



Insert Figure I about here 



The cognitive elements of the task serve as choice points in the task and represent the 
salection of a given path from several possible onesi they appear at nodes in the grammar 
from which several links emanate. I^te that the task analysis presented does not represent 
• elnglo letter typing task but the class of such tasks. Letters will differ depending on the 
toiections made at the choice points. The example { am considering is that of typing a letter 
on 9 standard office electric typewriter. The letter to be typed is illustrated In Figure 2. 
The copy from which the letter is typed can be assumed to be in reasonably finished form, 
simplifying this exampio. 



Insert Figure 2 about liere 



The first choice made m generating the letter selects the line on which to start typing the 
date. This depends on the size of the letter and the default option for typing this particular 
type of tetter. This is a cognitive action and follows the motor actions of inserting the paper 
end aligning it. In the case of this example, two motor elements describe the process, <set 
Spacing to 1> and <hit carriage raturn>*^ 

C^.,.^.ate date> fires the cognitive element to <retriaye date>, typically, from one's own 
momory. This, turn, fires the motor elements to position the paper end type the date: 

• 

<strike space bir>» 
<type month> 
<type dty> 
<type eomma> 
<fype year> 
<Mt carriage return) 

Theso eSements can be broken into further subeiements such as <type •!•> <type '9*>, etc., but 
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March 31. 1979 



Dr. JoiepH X. Smith 

Department of Psychology * 
Carrwgie-Melion thlvf^rsity 
Ptttsburgh, Pennsylvanii * /213 

Dear J&e« 

In regard to our telephone conversitfort of March 30, 
1979, 1 agree that now \s perhaps the byst time to 
purchase new minicomputer equipment for the NET4 
learning experiment I suggest that we looK at the four 
Vflmtors on the list I am enclosing. 
I wifi bo In Pittsburgh next Thursday to discuss these 
matters further with you. 



Yours sincerely, 
Anne Browne 

AB/ik 



Fisur« 2. Simple of letter modeled In ATN rramn^ar. 
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tMa will not fa« necessary for the ©xam^fe. 

Note that If a wrong choice Is made In selecting the number of lines to space before Inserting 
• date, tlie following badly formatted letters might result. 

1. A short letter squeezed at the very top of the page. 

2. A letter which fits on page one except for one or two lines, 
a A letter positioned too close to the bottom of the page. 

Any one cf th^se letters needs retyping. Therefore, that particular choice point Is crucial for 
task efficleney. A typewriter carriage which covers the untyped portion of the paper from 
view luakes the correct decision more difficult. A task representation of this typing situation 
-^ould Indicate the problem and Its cause. The task analysis which follows cn the computer 
Interaction tasks pinpoints similar cMce points which can then be used to determine what 
computer system environment characteristics generate a poor choice of task elements to 
follow. 

' The above typing task description illustrates what constitutes a task analysis for the 
particular form of computer interaction tiisks I am discussing. The particular task elements 
shown for the typing task are hypothesized} in the next section, I describe a methodology for 
obtaining task elements experimentally, and for generalizing from several experiments to a 
general case representatioa 

3. Performing the Task Analysis 

As shown in the previous section, the Analyses 1 perform on human/computer Interactions 
consists of breaking a task into subunits or what I termed task elements, I implied In the 
typing task analysis, that the selected elements were general, and that thi^y could be found 
occurring again and again In various pitrts of the task. Below I list my criteria tor an 
operation to be selected as a task element. 

1. The time to perform the task element should be relatively constant. 

2. The time to perform the task element must be within the resolution of the time 
measuring device. 

3. The task element must have a clearly observable start and stop point. 

4. Task elements must be Independent units of behavior. 

5. Each task element must occur more than once. 

6. A task element m\^s\ occur in more than one combination of other task elements. 

-5- 
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7, Th« prop«rtle» thai characterize a tasK element must be present In almost all 
inttitKCS of Hi occurrence. 

a The set of task elements must be exhaustive, that is, they must account for all of 
the observed stream of behavior. \ 

Tho taek under consideration In this paper Is a computer-aided Instruction task. The user k 
ioeted In front of a computer terminal. Text and questions about the text ire presented to 
iho user. The user responds to these jestlons by Keying in a number for a selection from a 
iitt of possible answers displayed to him or her. Examples from this Jask are used to clarify 
tho eteps In the algorithm I am presenting for performing i task analysis. 



Insert Table 2 about here 



The bottom up execution of this task analysis is described in the steps listed below. Task 
goals end environment characteristics provide top-down constraints for building this 
ropresontation. Table 2 Is a synopsis of these steps. Below is a description of the steps to 
teke In performing the task anaiyslsj these steps are expanded further In terms of the 
computer learning task being analyzed in section 5. 

Stops for Performing and Analyzing a Task Anstyiis 

1. Select a simple example of thb 4sk to be studied. If the task is learning from a 
computer display, select a string of three or four displays that characterize the 
entire learning 

2 Make a record of an Individual performing this task either on videotape, on 
' eodiotape, via cemouter recorded keystrokes, by manual observiition and 
note-taking or by any combination of the above. Elapsed time indicators must be 
included in any such record. 

3. Mark the stream of befeavior with task deUmittrs. T»sk delimiters an| changes In 
* the behavior stream} what changes will be observable depends partially on the 
recording medium. For a task with multiple observing methods, several delimiters 
will overlap each other. The process of marking the task delimiters occurs in the 
following order. 

- Mark all cues that govern the task. If the task requires page turning, then 
each page turn start is a delimiter. If the task requires reading, then text 
propositions and paragraphs are delimiters. 

- Select alt motor operations that have distinct start and stop points, tf the 
operation Is typing a key, then the start delimiter occurs when the finger 
touches the desired key. The stop deilmiier occurs when the key rises to 

-6- 
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1. Select represcntitive task. 

2. Record t«sK behavior. 

3. Mark ctetimiters in recorded behavior. 

4. BreeH task into units. 

\ 

5. Assign units to behavior classes. 

6. MarK task goais on recorded behavior. 

7. Combine units into new units until only one remains. 

8. Modifiy task and redo grammar induction, adding new details. 

9. Calculate ttw amount of time each path takes. 

10. Search for paths with long execution times. 

11. Generate and test hypotheses for long execution times. 

Table 2. Steps for performing and analyzing a task analysis. 
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tha Initial pofJtlon Again, 

- Select all changes l.i the environment as a ta&K delimiter. If the corr.puter 
finishes a previous task and Indicates its ccmpletion by displaying^ 
flashing- asterisk, this display serves as a task delimiter Jhe ePPe«rarAe 
of another character (possibly typed by the user) on the display scraen 
marks still anottwr task delimiter. 

. ^oo^ for verbal cues sucri as statements of task completion or commentary 
that indicates the end of a sequence or the start of a new or*e. Comments 
like There- or "Now, let's do" or "Yes, that's what I want" serve as verbal 
task delimiters. - . - . 

A. Break the task under consideration Into units based on these delimltef s and label 
these units. If the stream of behavior is a verbs? record and contains, ♦he 
comment "Oh. new page" followed by the comment f° ^j^'"* 

to do here." the unit of behavior between these verbal delimiters Is a likely 
candidate to be labeled <turn pageX This ja a sfmple examples other labels can 
Often be <A!> <A2>, etc. because missing delimiters or too many delimiters make 
it difficult to perceive what behavior is taking place. 

5. Once the units of behavior have be^^n labeled, determine t^e amount of time it. 
takes to execute these elements and calculate their descriptive statistics, i.e., 
me^ns. variLe and frequencies of occurrence. Use these s atlstics u help 
S If units are members of the same class. Attempt j'«7"J^ 
that appear as outliers by grouping them into larger elements or breaking them 
into smaller ones. 

6 lay out the goals of the task being executed in as detailed a form as possible. 
iTrk "hese goals on the record of user behavior. Mark higher level goals with 
stronger weights than lower level goals. 

7. Form groupings of two behavioral units bas.-d on their co-occurrence. For 
example. If one type of element follows another in several cases, consider this 
sequence a new group. Calculate the p > ^alililies of one unit following a second 
etS^JId thrSw out those groups below « set threshold level. If groups overlap, 
find those Instances where the overlap occurs together in the behavioral stream 
select the grouping which occurs first. If there Is no overlapping, retain 
' both groupings. Rename these groupings .a« higher level elements. 

a If an element follows Itself, i emove one of the elements from the stream of 
behavior, but indicate the repetition on the remaining elfnent.^^^ 

9. If .groupings overlap a goal marker, the^T are required ,^«P^ff^^^ 
the beh^or between adjaceni goal marks of equal weight. .^Mhis l^ot the 
case, redo the groupings, recombining the elements w.thm each go^i .t*.t. Only 
.cross the goal boundary after such elements have been combini^ « * 

10. Repeat steps 6 through 8 until a single remaining group exists ^'^^"^/'"^ 
a structure that forms the first st^ge of an augmented transition network 
grammar. • Rewrite this structure such that its nodes are states and its arcs are 
actions that pass the elements that combined to form a given state. 
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11. fterun tfie prototype tash with str.aii modifications and different subjects. !n each 
cese, executs steps 1 through 8, generating a new structure. Compare each 
structure l<»king for differences and resolving thiese differences fay expressing 
■II possibilities in a larger ATN representation. For example^ if subject 1 exhibits 
behavior <A> <B> and subject 2 exhibits behavior <A> <0, the resulting 
expression shiould look like Figure 3. 

12, Caicuiale from the experimental data the average probability of occurrence of 
each arc In the ATf^ f,rammar. Select every path In the representation and 
celculete the amount of time it would take to execute using thie probabilities to ^ 
obtain values for paths that have multiple choice points. 

la Using the times for these paths, at each level in the grammar, lock for paths 
which take a long time to execute. Those paths which rerasent long completion 
4ime$ point to possible areas of difficulty with the user interface. 

»i4. Once possible trouble areas have besr. selected, mark tl^ h^ootheslzed reasons 
for various path selections on the arcs. Use these semantics to suggest different 
interface designs that would lower the path time. 



Insert Figure 3 about here 
^ 

At this point, the ATN grammar has been produced and labeled with time to execute values. 
Linear models of an individuars behavior can be built by selecting various paths in \ho 
grammar. Tnes? can be verified by putting ♦he Initial data for the time fO execute for each 
task element into a linear regression equation. 

The grammar can be refined by additional experimentation using other tasks and subjects. In 
addition tasks not yet undertaken wilh the particular computer interface can be broken down 
in't ttie elements already expressed in the grammar. Their ordering can be hypothesized and 
a prediction generated of Expected task completion times. Thus, the representation serves 
not only as an analysis tool but as a generator of possible behaviors In new situations. 

4. Th« ZOG System 

Before beginning a detailed example of the task analysis I am describing, I will briefly present 
a description of ZOG, the computer system on which the behavior measures take place. ZOG 
is described In dot ail elsewhere (Robertson, Newell, & Raniakrishna, 1978; McCr acker. & 
'Robertson, 1979)? I present only the essentials of its operation here. 

ZOG is a communication agent. It Is used te aid information flow between an Individual seated 
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•t a computer terminal and a ZOG database. The computer aided instruction tasK occurs In 
this mode. 

ZOG databases have a unique structure. The unit of storage in the database is a computer 
terminal display of text. Each of these displays is called a /rama. Figure 4 Illustrates a 
aampte ZOG frame. Each of lite frames in a ZOG database may be linked to any of the other 
frames, forming a complete network. Such a network Is termed a eognet. A link Is 
raprasented as a numbered selection on a frame. When a current frame Is displayed before a 
user, any one of the frames to which this frame is linked can be brought before the ur.er 
when he or she types th» number associated with a selection. The text for these selec'lons 
briefly describes the contents of the linked frame. This is Illustrated In the five selections on 
figure 4'$ example frame. 



A series of selections by a user will present an equivalent number of displays to the user. 
This sequence of selections is called a path. The user has commands available at the bottom 
of the screen display which allow him to back up on the path he has taken, jump to a new 
frame not on the currer.5 path, or view Ihe context of his path within the zognet he is 
traversing. Users are able to change a zognet by building new frames, editing existing ones 
or changing links. ZOG is a menu-based display systemj ail command options available to the 
uaer are displayed before him as a selection menu. ZOG*s primary operation Is the display of 
information. 

S. Modeling a Solitaire learning Session 

The task being modeled Is that of an Individual learning how'io play a game of solitaire from 
• aet of rules and questions presented to the user with the ZOG system. The card game is 
the game of Golf. 

Each sentence of the set of rules appears on a single ZOG frame. The subject reads the rule 
aruil types a V ^° obtain the question display. The question display contains « question about 
the rule or an integration of the rule with previous card game knowledge. Each question has 
two to five possible responses, only one of which is correct. Figure 5 illustrates a sample 
rule and question frame for this task. If subjects choose an incorrect answer, they are 
diaplayed an error frame which explains the correct answer. They then type a V to continue 
on to the next rule. 



Insert Figure «about here 
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Stcndird Statistical Packages 

ThU net descfibas t»w basic statistics methods required 
of most psychologists. A brief description of ttw 
underiylng theory for each of the methods is fottowed by 
a list of the packages available for performing tfw 
■n«tysls. 

1. Descriptive Statistics ^ 

2. Correlations 

3. Analysis of Variance 

4. Factor Analysis 

5. Multidlmensionai Scaiing Analysis 



beck edit find goto help info mark return 



Figure 4. Example of a ZOQ frame 




1 



Marlfyti Mantei 



Modtiins Ustr BtHavior 



2 April iS79 



Ir»ert Figure 5 about here 

B«for« {eiMhg iho game of Goif, subjects pass through several frames teilfng them f\ow to 
respond to tNe CAl francs. They answer qiwstions about their previous card game 
knowledge, take a short reading spec^ test on-line, and learn a simpler solitaire game. After 
fMsl^ng all rules and <^uestlon$ about Golf, subjects are i^liowed to review the rules In 
consolidated form Following the computer-aided instruction (CA!) sesston, subjects are given 
• Ckio^ of qards and asKed to play the game. They are advised that 4his wilt take piece 
iMiforo they fteri the session. CorKeptual and strategic errors in playing the game are rrated. 

The record of continuous behavior for this task 5s a set of computer recorded keystrokes. 
THs is supplemented by knowledge of the task structure and goals. Because the data record 
contains only kityslrokes, measurement resolution is lew (approximately the. time It takes to 
road a ZOG framp>. A task analysis can be performed with this low level of resolutions if 
future measures, such as eye movement data, are of higher density, they can be incorporated 
In IhiR existing model by expanding one of the terminals of the grammar. The ATN grammar 
rosultlng from this low-level data stiil provides several insights Into the CAI task. The results 
from the analysis of the grammar are nof astonishing, but serve to Illustrate the methodology 
end tt$ possibiiitiet.. 

The obvious delimiters in this task are iwystrokes. <Raad ruta> elements are terminated with 
V> <read qu«stion> elements are terminated with a numeric keystroke and <raad •rror> 
•tenwnts terminate with a V. Table 3 illustrates a partial computer record of a subject 
loarning the solitaire game. The keystrokes and the Interface elements of the ZOG system 
are used as task delimiters. Column 1 of Table 3 contains hypothesized element names that 
are assigned to the units between the task delimiters. Columns 2 and 3 list the start and 
atop delimiters of the task, and column A gives the associated time for the task elements. 



Insert Table 3 at>out here 

Note In Table 3 that more than one 'q'» *c* or number is sometimes typed following the display 
of the next frame. The subject In this experiment is a heivy-flngered typist who depressed 
the terminal keys firmly and held them depressed for approximateiy one second. Becausp the 
terminal and ttre ZOG system allow typt a/iaad, this key depression was interpreted as new 
keystrokes. The first V recorded caused the question dispiay to appear. Additional q*t were 
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One card it at Ime may be moved form the top of a 

tableau pHe, If It Is In sequenca with She card on top of 
pm talon. The sequence mty be up or down. 



pUESTlON; Which of the following card sequences cannot 
be placed one at a time on top of the talon? 

1. 6-7-8-7-8 

2. A-2A 

3. J-10-9-9-8 

4. Q-J-Q-K 

• FIsiire 5. Example of CAI f rsme. 
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Etement Slarl cteHmiter ^top delimiter tlapsed time 





^ta*a%ltt%# fvswtaft 

Mispisy irinw 


M 


If) ^ 




afspisy ffsfffo 


M 




9* <wiii qiiesiion> 


beep 






•I. <W«lf C)U0StlO?V 


Deep 




U.vc 


51. <wtit qu9^iion> 


Deep 






.beep 






7a <wait f uis> 


dispiey irame 


^* 

a 




8. <type < 


beep 






9. ^craad qi»&tion> 


display frame 


I 




10. <walt arrof 


display frame 


1 




i i. <walt error* 


Deep 


i 




iZ. <1ypa 


beep 


c 


Oil 




Wl9^i«jf If Vif^P 




0 98 


14. <wait rufe> 


beep 


V 


0.03 


IS. <wait rule> 


beep 


V 


0.33 


16. <type •q'> 


beep 


V 


7.70 


17. <raad question* 


display frame 




19.03 


18. <r99d rule> 


display frame 


V 


18.72 


19. <wali qi»stlon> 


display frame 


V 


0.67 


20. <waH question* 


beep 


•q* 


0.03 


21. <wait question* 


beep 


V 


0.23 


22. <wiH question* 


beep 




0.03 


23.<type'l'> 


beep 




7.38 



Tabl« 3. Example of computer captured record of solitaire learning tasie 
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Ktorllyn Ktont.l ^ Mo^'i-t B"""'" ' 

co«>.d,r,d .n error s.l.c.lon <b.c.u.. «, V s.l«tlo" -xLfO '""f ' 'l^T. 

!vrf.« responded with en eudlb.e beep to indic.te . mistyped Key. The t.sk e .n«nl. 

o7h ri„on.l Key int.rpret.tlon by the system .re <w.« ,u..«.n> <w.« error 
rS?rirThey c.n slow the display by up to two seconds, but their prese.ce does not 
«|gnlflc«ntly affect overall subject performance. 

„ no type .t»«J occurs, frr e««p.e, In lines 2, 17 and 18 o. T.ble 3, the t.sK element Is 

L^or^^d .s <r.«l rul.> or <r.Kl ,u..«on>. representing the time to process . rule or . 
Interpreted .s <r.^r«. ^^^^^^^ ^^^^ ^^^^^ ,„ 

:::,r. trj ; wr..mpora.y d,sc,rd the -..1 o, ^u...... ^c. 

^w^er. since-^tne is the cr^ lysis n,e,svr. in performing . tasK ""'V'";. 

^^.. Ji ed each <r.«i question* <r..d n,l.> and <re.d .rror> element by the .mount o time 
rllulred for their execution. This classification is indicated In T.ble 4 .long w»h .n 
irpUn'tlon o, the hlsher level elements that are formed from <r.«t ,«..t.on> <r.«. r«l.> .nd 
<r««d •rrQr> combinations. 



Insert Table 4 about here 



Using the methodology described in section 3. higher level elements are bull, -m th^ < 
^ .nd <r.ad qu..tion> elements. Their combination is controlled by the goat o' '^e t.sK 

to iTe gar,,e of Golf, this goal Is broKen into subgoais of 
rrg.me Quite natu.aliy. these are further split Into the goals o, acquiring '"'^^^ 
Z Lorm.non presented in each frame. Figure 6 shows both ^f''' J" « J^^/,^^^^ 
.7pIlc.tlon to the sequence of tasK elements in the behavior record. The sequence o. tesK 
•lement* Is taken from the behavior record shown in Table 3. 



Insert Figure 6 about here 



UV.1. of d.i.11 for the goal, end t.sK ele^nts .re Indicted In Figure 6. Eech t^lgher 
i^mblr B0.1 level Is . set of subgoais of th. previous level. The level, for the t.sK .lament, 
rlur They represent the combination of loweMe.l 

tr.t.ulH.; once a. the boundary level, the o,^ cons,r.int on combining element. .. 
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Tff k Etewcnl gypUnatten 



<rtH« i> 
<rul* 2> 
<rute 3> 

♦ 

<qiMst 5> 
<8rror 1> 

3 

Mirror S> 

<RQ> 

<C^> 

<Qe> 

^Q*> 

<QEQ*> 

<QeQ> 

<Q0'> 
<QC)> 



Rtad rtiU 



in< to s§eonds 
in tO'20 seconds 
in 20-30 s€con4* 
in 30-40 seconds 
in> SO seconds -. 



/?corf questions aecorcUng to ob^e times 



Rmul VTors according to tU»eve times 



Rend ru/« and answer question 

Read nde, answer question and read error message 

Learn set ef rates 

Learn two rules making tr?w on second question 
Make error on first rule* then none on rest 
McJte error on second rule, then none^on rest^ 
Learn rale, make two errors, then none foUdwing 
Ltforn rates vfUh some errors iniermingUd 
Learn game of soUtaire 



4. Meaning of task elements for solitaire learning task. 
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■1 





Aim s 2^ 


wet 4 \ltV£L 3 zlVfi 4 ifv^£^ i 


UMtl HiftU, MAP 

ISAfyt/ifMI^ ASAP fAm^J 
HeApfA^fV£$ 


</Wii«> <A<3> «?6> <9e<?«> 

<g/f/0^5> <£> 

<MUi> <(iq*> <A^> <^?»> 



fmo^£ €. f4SK 6l€^€NfS /M 9i/9/eCf$ UAHHIit& SOUfAt^C tM ZCO, 
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Ihsir high probability of co-oeeurrence. 

In the solitaire gamo, <pead rule> followed by <reail quattien> occurs most frequently, so • 
n«w element <nQ> is formed. Although <RQ> now follows <RQ^ more frequently than any other 
element, the goal boundaries for learning rules are not to be crossed until the <read error> 
clement is combinedi therefore. <RQD is formed from <RQ> and <read error). This brings us 
to tevel 3 where we can combine strings of <RQ> to form <Q*>. <RQ> and <f?QD are all that 
remain and |re combined to form <QE>. Elements <QE> and <Q»> then form the single start 
ttete of the grammar, <QEQ*>. Table 4 gives an explanation of each of these states. 

The grammar formed from tNs elements in Figure 6 is an illustration of ATN grammar 
generation from the behavior record in Tcble 3. Figure 7a illustrates the grammar that 
l-eaulla from the entire record for this sar^ subject. Since subjects made different errors In 
answering questions, the grammar resulting from running additional subjects In this task 
becomes more general. This Is shown In Figure 7b. 



Insert Figure 7 about here 



Ttie ATN representation in Figure 7 is shown in a non-standard form to provide clarity. The 
etements should be Information that is passed along the HnKs, and the nodes In this 
representation are slates from which several links emanate. Which path Is taken, depends on 
the conditions that exist when an individual is in that particular state. The rules for which 
peth to take are the semantics of the ATN rapresentation. At present, no semantics are 
applied to the grammar that has just been built. The purpose of the representation is to find 
paths in this grammar which take a long time and then to hypothesize why these paths were 
executed and to generate task environment changes forsaking them shorter. 

The choice points of the ATN grammar for the sjlltaire learning task are rather simple. The 
two main ones diverge to <R0> and <ROE> states. Five subjects were run on this task of 
readif^ and assimilating 12 rules, 12 questions and possibly, 12 error messages. Out of the 
60 questions that were asked of the subjects, 20 or 33Z of the answers were in error. The 
everage time for execution of <re»d arror) was 9.43 seconds, adding 37.7 seconds to each 
subject*s time. The average time for a subject to read 12 rules and 12 questions was 342.2 
eeconds. The addition of the error processing Increases the time to do the task by 11^ The 
error link Is a trouble point In the CAl task. This pinpointing of rewJIng error messages as a 
trouble spot in the task is an obvious result obtainable by simple statlsticsf the result, In this 
case, was obtained by using a small but general sot of heuristics. 
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<QQ> ►<qEQ»>-*<QEQ> 

<QEQ0-^<QE> <(}•> 

<PIQ> — ► <QEQ«>— ^ <IQ*> 
<EQ0 — <^<J£> — ►<Q«> 
<QE> ^ <^^> ► «VQE> — ^ 



«),> — KRq> ► % 

a 

<(19> ~^ <^^5 ft^i^> <a6A?> QgesT*o'^> 

Fl<WHc7fiL. Au<r/vJ€fJr€S> T^ArtSltlO^l aI«TVJ<5R»^ 4u8j€€T 





fl6tm£7k flu<^weMffc3> •mftfJSlTIOM NSXUJORK GRAMftftfl f<A SuSJtcTS 1-5. 
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A traiKl consideration In looking at the cost of reading error frames is that of the 
heavy-fingered typist. Given a different CAl interface where alt selections were nunfibers, 
fh« unijxpected type ahead would have caused numerous error frames to" be selected, plus 
too fast a display of the individual frames for learning purposes. The high cost of such a 
patfi would indicate a need for a system design change. 

A second variation of the solitaire learning task was tested on six new subjects. In this 
second task, the rule and the question about the rule are presented on the same frame 
display. The resulting ATI^J grammar built from the behavior records it shown In figure 8a. 
Note thai' more qi»stion and rule frames occur between the occurrence Of error frames In 
this representation. 

Insert Rgure 8 about here 



In this second task, errors are not a serious problem. Nine errors occur for 6 subjects 
onswering 12 questions each (137. errors). The average time to read an error frame In this 
task was U,9 seconds. The average task time for each subject was 316.4 seconds so that an 
average of. 17.9 seconds spent on errors for each subject adds 5.6 percent to the time to 
execute the task. If the grammar in/Figure 7b is combined with that of Figure 8a, the 
grammmar shown in Figure 8b will result. The decision applied to the Inltiaj diverging links Is 
^ther the rules and questions are combined or separate. The combined path Is the least 
expensive one. This result compares to that already known In the CAI literature (Holtzman, 
1970), but again, It fails cleanly from ttie methodology. 

If we examine paths in the grammar at the less aggregate level, we find that some rules take 
longer to re.ad than other rules. The resolution of the daia does not allow us to make many 
hypotheses about the causes of these differences except by looking at the number of words 
or propositions In each frame. Those frames for which longer times are explained by text 
tongth are considered a necessary part of the task, but outliers from these explanations can 
be looked at as possible trouble areas. This is a suggestion fpr deeper analysis of the data 
llslng.tHe^molhodology being presented 

6« C^ondution 

By bultding an ATN grammar from the low resolution data obtained from this task, two 
problem areas were notedi the first was that of excessive subject errors In the learning taski 
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<flQ> •^<QQ'> 



/ 



<KQl> 



^ Fiau^i 'Sd. AT^ ^^^^^ f"^^ ^^^^ ^^/fST-ztf/N/. 



^<QQ> — '^<KQy 
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4he second was an Interface problemg|ti the type aKad facility. A resolution to the error 
problem ceme fro^ ntedifying the ^slightly. The type ahead problem w^uld not have 
shown up as a significant difficulty If M had not been generated as a task element In the 
behsvlor- record and If the bad effect of errors had not. been noted 

The CAI task was s|^cted for presentation in this paper because of the simple grammar that 
resulted and the indications of problem areas that were Important and occurred at high 
levels. Application of the methodology dt scribed in this paper to more complex tasks such as 
computer text edll^ or bibliographic retrieval would investigate such areas as subject 
strategies and the effect of computer command structures. InJ,hls environment, the resulting 
grammars would be very complex and expected to discover Interesting problem areas. This 
methodology works best when the task Is well-structured. Without goal str ictures to apply 
to the task, little control' Is exercised over the grammar buildin'g resulting In Incorrect 
^ representations of the task. 

' If • regression analysis is performed or- ♦he same task element data, Information |s,!ost about 
the exac^ change point in the proc.s where less cost effective paths were taken. 
Regression analysis can be used in conjunction with the ATN grammar building. For example, 
{f . set of causes are hypothesized for a path taking a long time to execute, s regression 
equation can Indicate how important these causes are^Large residuals from a regression run 
can also be used to Indicate new problem areas. 

The methodology I am proposing is still in its developmental Infancy. It has not been tf^led on 
• large variety of tasks or subjects. There ^re problems of order and structure to be 
^ considered In the induction heuristics. Until n lar^e variety of tasks have been tested, the 
generality and validity is questionable. How n<uch structure is required of a task in order to 
perform the grammar induction Is not known. The results from the computer aided instruction 
task took promising. 
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